Skip to content

fs: add io_uring open operation #7321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Conversation

mox692
Copy link
Member

@mox692 mox692 commented May 8, 2025

Overview

This PR implement io_uring version open system call.

Since common file operations, like fs::read and fs::write, use open under the hood, I think suppporting open sysacall first is a good starting point.

Also, since this is the first operation we actually ship, I've added:

  • Some uring fs tests
  • Runtime flag for enabling uring (enable_uring)

Implementation

  • When constructing OpenOptions, the implementation is chosen based on the presence of cfg flags.
  • The only difference is that its open() method, which uses io_uring or thread_pool.
  • Even when calling open() on UringOpenOptions, if the machine does not support io_uring, it will fall back to using the standard OpenOptions.

Much of the code in this PR is ported from the tokio-uring crate.

Why do we need our own UringOpenOptions?

The standard library’s OpenOptions does not expose accessors for its internal flags. However, when using io_uring to perform the open system call, it's necessary to access these internal flags. Since this is not possible with the standard implementation, we define our own UringOpenOptions type.

This limitation is tracked in the std (rust-lang/rust#76801), but it has not yet been resolved.

@mox692 mox692 added A-tokio Area: The main tokio crate M-fs Module: tokio/fs labels May 8, 2025
@mox692
Copy link
Member Author

mox692 commented May 11, 2025

I ran a benchmark in commit d0dc6b2.

settings
$ uname -a
Linux mox692-ThinkPad-P51 6.8.0-59-generic 61-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 11 23:16:11 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

# On commit `d0dc6b2`,

# uring bench
$ RUSTFLAGS="--cfg tokio_unstable_uring" cargo bench open_many_files

# thread pool bench
$ cargo bench open_many_files

Single-threaded runtime

Benchmark Name Thread Pool Uring
open_many_files [5.0981 µs 5.1226 µs 5.1520 µs] [4.1279 µs 4.1374 µs 4.1479 µs]

Multi-threaded runtime

Benchmark Name Thread Pool Uring
open_many_files/2 threads [3.5071 µs 3.5448 µs 3.5911 µs] [4.0503 µs 4.1464 µs 4.2529 µs]
open_many_files/4 threads [3.2188 µs 3.2382 µs 3.2582 µs] [5.0549 µs 5.1624 µs 5.2627 µs]
open_many_files/8 threads [3.2208 µs 3.2434 µs 3.2663 µs] [6.6902 µs 6.7288 µs 6.7705 µs]
open_many_files/16 threads [3.4278 µs 3.4474 µs 3.4675 µs] [7.4527 µs 7.5142 µs 7.5674 µs]
open_many_files/32 threads [3.9230 µs 3.9604 µs 3.9977 µs] [6.8199 µs 6.8953 µs 6.9600 µs]

While performance improves in a current thread runtime, the results actually regress as the number of threads increases. Currently there is only one global ring in the runtime, so increasing threads may be causing lock contention.

I'll look into whether I can reduce the lock contention.

@Darksonn
Copy link
Contributor

The main feedback I have here is that although it's ok to use #[cfg()] temporarily, eventually we will want --cfg tokio_uring to result in a binary that supports both io_uring and spawn_blocking. I.e. it should fall back to spawn_blocking if the Linux kernel it runs on does not support io_uring.

Base automatically changed from mox692/iouring_initial_infra to master May 20, 2025 17:36
@mox692 mox692 changed the base branch from master to mox692/uring_dynamic_check May 24, 2025 09:38
@mox692
Copy link
Member Author

mox692 commented May 24, 2025

Update: Sync with master and now the base branch is 7357.

Base automatically changed from mox692/uring_dynamic_check to master June 10, 2025 18:26
@mox692 mox692 marked this pull request as ready for review June 15, 2025 16:10
@mox692
Copy link
Member Author

mox692 commented Jun 15, 2025

This is ready for review.

Copy link
Contributor

@Darksonn Darksonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks reasonable to me. Have we verified whether CI runs a recent enough Linux to actually run the io_uring code?

@mox692
Copy link
Member Author

mox692 commented Jun 20, 2025

Have we verified whether CI runs a recent enough Linux to actually run the io_uring code?

Based on here, ubuntu-24.04 (which is equivarent to the ubuntu-latest) is based on the kernel 6.11.0, so I think the current CI is running a kernel new enough to use io_uring.

Also, I've added a tiny verification step in the CI in 2ec8482.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate M-fs Module: tokio/fs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants